It will vary from machine to machine.
After a while perl might run out of memory if
you supply it with too many pages, but before
that happens, you will probably notice it
takes too long to search your site. When
conversing with one user of the script, after
they had asked if my script could search
through 16,000 html files (7 MB total), I
asked how much the script could handle. Here
is the response:
Your doubts were well founded. I
tried it as an exercise; after a few
hundred, things start slowing down. With
about 2,000, not worth mentioning.
-- Robert Savage
<rjs@wmw.com>
Use the script within reason. It runs
fairly fast on my site, and I have it
searching about 400 - 500 KB in files. It all
depends on the speed of your machine and how
long you think users are willing to wait.
